Additive Logistic Regression a Statistical View of Boosting
نویسندگان
چکیده
Boosting Freund Schapire Schapire Singer is one of the most important recent developments in classi cation method ology The performance of many classi cation algorithms often can be dramatically improved by sequentially applying them to reweighted versions of the input data and taking a weighted majority vote of the sequence of classi ers thereby produced We show that this seemingly mysterious phenomenon can be understood in terms of well known statistical principles namely additive modeling and maximum likeli hood For the two class problem boosting can be viewed as an ap proximation to additive modeling on the logistic scale using maximum Bernoulli likelihood as a criterion We develop more direct approx imations and show that they exhibit nearly identical results to that of boosting Direct multi class generalizations based on multinomial likelihood are derived that exhibit performance comparable to other recently proposed multi class generalizations of boosting in most sit uations and far superior in some We suggest a minor modi cation to boosting that can reduce computation often by factors of to Finally we apply these insights to produce an alternative formu lation of boosting decision trees This approach based on best rst truncated tree induction often leads to better performance and can provide interpretable descriptions of the aggregate decision rule It is also much faster computationally making it more suitable to large scale data mining applications Department of Statistics Sequoia Hall Stanford University Stanford California fjhf trevorg stat stanford edu Department of Public Health Sciences and Department of Statistics University of Toronto tibs utstat toronto edu
منابع مشابه
Additive Logistic Regression : a Statistical View ofBoostingJerome Friedman
Boosting (Freund & Schapire 1996, Schapire & Singer 1998) is one of the most important recent developments in classiication methodology. The performance of many classiication algorithms can often be dramatically improved by sequentially applying them to reweighted versions of the input data, and taking a weighted majority vote of the sequence of classiiers thereby produced. We show that this se...
متن کاملEvidence Contrary to the Statistical View of Boosting
The statistical perspective on boosting algorithms focuses on optimization, drawing parallels with maximum likelihood estimation for logistic regression. In this paper we present empirical evidence that raises questions about this view. Although the statistical perspective provides a theoretical framework within which it is possible to derive theorems and create new algorithms in general contex...
متن کاملIdentifying Risk Factors for Severe Childhood Malnutrition by Boosting Additive Quantile Regression
Ordinary linear and generalized linear regression models relate the mean of a response variable to a linear combination of covariate effects and, as a consequence, focus on average properties of the response. Analyzing childhood malnutrition in developing or transition countries based on such a regression model implies that the estimated effects describe the average nutritional status. However,...
متن کاملSmooth ε-Insensitive Regression by Loss Symmetrization
We describe a framework for solving regression problems by reduction to classification. Our reduction is based on symmetrization of margin-based loss functions commonly used in boosting algorithms, namely, the logistic-loss and the exponential-loss. Our construction yields a smooth version of the ε-insensitive hinge loss that is used in support vector regression. Furthermore, this construction ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1998